Plenoptic cameras are gaining a lot of popularity in the field of computational photography because of the additional information they capture compared to traditional cameras. Indeed, they are able to measure the amount of light traveling along each ray bundle that intersects the sensor, thanks to a microlens array placed between a main lens and a sensor. As a result, such cameras have novel post-capture processing capabilities. For example, after the image acquisition, the point of view, the focus or the depth of field may be modified.
A schematic view of a plenoptic camera is represented on FIG. 1. The plenoptic camera 1 is formed of a lens arrangement associated with a photosensor array 13, also called imaging sensor. The lens arrangement comprises a main lens 11 and a microlens array 12, which comprises a plurality of microlenses (e.g. an array of L×L microlenses). A micro-image is an n×n image formed on the photosensor 13 array behind a microlens. The collection of micro-images forms a 4D raw light field image more simply called a raw image or a raw plenoptic image. From the raw image, a matrix of views (also called a set of sub-aperture images) may be obtained by demosaicing and demultiplexing. The demosaicing enables to recover a full color raw image, i.e. to recover full color information (for example RGB information, RGB standing for “Red”, “Green” and “Blue”) for the pixels of the raw image. The demultiplexing performed after the demosaicing enables to recover the different views. Demultiplexing comprises grouping co-located pixels from every micro-image after demosaicing. A view is also called a sub-aperture image. Some cameras such as the Pelican Imaging® camera, directly deliver matrices of views (i.e. does not require de-mozaicing). Moreover, the term 4D light field image (also noted plenoptic image) corresponds to either a 4D raw light field image or a matrix of views (see for example the Chapter 3.3 of the Phd dissertation thesis entitled “Digital Light Field Photography” by Ren Ng, published in July 2006).
As depicted on FIG. 2, in the case where the microlens array is an array of L×L micro-lenses and each micro-image is of size n×n, then the matrix of views obtained after demosaicing and de-multiplexing is of size n×n with each sub-aperture image/view being of size L×L.
Encoding such content (i.e. the raw image or the matrix of views) with known video codecs (e.g. H.264, HEVC, etc) is not optimal because these codecs do not take into account the specificities of plenoptic imaging.